DOCUMENT RESUffE 



ED 243 905 
TITLE, 

INSTITUTION _ 
SPONS AGENCY 
PUB_DATE ' 
NOTE 

Ptlf'-TYPE, 

"EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



TH 8^0 843 4 



Archiving Methodology, ^olume II; Contractor's 
Guide . _ * - ; 

Le in wand (CM.) Associates, Inc., Newt on, Mass.' 
National Inst/ of .Education 1 ED) , fjashingtpn, DC. 
31 Mar 80 _ ~ * 

29p. { For related documents^ seeTM 830 "842-845; 
Reports - Research/Technical (143) 

MF01/PC02 Plus Postage., 

*Archj.ves; ^Databases; Bata^Collect^on ; *Belivery 
Systems; Documentation ; Guidelines; Information 

Utilization % '_ 

^Contractors; Database Producers; *Data Organization; 
Secondary Analysis • * 



ABSTRACT . 

This section of t»e_ "Archiving Methodology " is 
concerned with the role of data collection contractors in the 
archiving process. Their role is discussed from three perspectives: 
(1) the nature of the relationship 6£ the data collection contractor, 
the project officer, and the archivist; (2) the considerations 
contractors must make when preparing data f or^archiving^ Sjid (3) the 
mechanisms facilitating the transfer of information from tfee 
contractor to the archivist. This document discusses an overview of 
contractor requirements (project-and file-level documentation) and 
specific contractor requirements (Cross-Referenced Information Fcfrm 
^ahd data cbllftetibn process ^rtStisiderations ) . (PN) 
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1' ' VOLUME II 

GUIDE FOB s THE DATA Bi 



This section of the Archiving Methodology is concerned with 'the role, 
of data- collection contractors' in the .archiving, process. 'Their "role "is dis- 

cussed from three perspectives: * - 

* _ : * ■ - ' • 

• the nature of the relationship of the • y . • 

. §ata collection contractor i the^projfect - v 

officer, and the archivist; ^ » 

-* * • 

• the considerations Sohtrectbrs piust make when' * 

preparing data for archiving; 1 * \ - 

• the mechanisms facilitating the transfer of 
information from the contractor to the archivist. * 

As Volume I indicated, the project officer decides what should be archived ,* 

... * _ s* 

how much effort should be devoted to archiving it, and how the- information • 
in the final product (archive) . is to be. disseminatedl The contractor and, 



archivist perform their work according to these decisions; the final product- 

* « v . ; 

-the "data archive— conforms to the guidelines the project officer has established. 



- • . . • i 

, • • • » / * 

7 ■ t . 



The archivist depends on the contractor to provide- study ^.nf ormation -sufficient 
to document the study data. ; • . 

Preparing an 'archive' depends on a reciprocal relationship between the' 
data collection contractor and .the archivist. Like any good working relation- 
ship, this relationship -'demand** communication, between the parties involved 
-....Si: . . • : i * 

The archivist stipulates the kinds of products required from the contractor 

• ... . _■ » *. ; \ . _ • * 

for the creation of usable and appropriate documentation, in tur», the products 
. the contractor delivers must be sufficiently detailed and- comprehensive to 

describe ,the data collection effort adequately. To facilitate this cooperative. 
^ effort, we will discuss the materials t^e archivist generally requires -from 
the contractor. These include copies of all reports and tapes generated 
, by the study. \ ■ ' 

Two issues are fundamental to understanding contractor requiremerils. s * * 
First f- the materials needed are those expected from,, and generated by, any 
sound research and reporting activity. Second, the recording' of the necessary 
information £akes place throughout the course of the study. ; 

The archivist does not ask the contractor for materials and ^formation = 
peculiar to the archiving endeavor or superfluous to the research process. 
For example,^, in requesting a 'detailed sampling plan including modifications * • , 

, made proportions of the target population obtained/ the archivist is 

* • * 

"j t , as king for information that vitally affects .the data.. Normally, the contractor 

records this information as the study is being planned and as data collection 
► . proceeds, and later; incorporates* it in the study reports. The contractor 
' provides this information for tfce same* reason the archivist desires it: ' 



the data are affected by sampling design and obtained population. Likewise, 

Voders of the study reports and users of a data archive seek this information 

. ^ . . . _ . _ f 

for the same purpose: to .understaiidcwhat the data represent., 
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■ In reality, studies do hot always meet the -ideal of found research and ^ 

reporting practices ; To the extent -that a^Study departs froi 'this - ideal , '* i 

the archivist must request "additional" products fro^tne contractor. However , 

in such cases', it is Clear the archivist is.helping the contractor more completely ' 

report the research conducted. As a result , t«he archiving process will affect 

the study beneficially. If contractors know the data- generated by their * ' 5 < 

studies^are to be archived, they will be encouraged to maintain logs and 

records accounting for study design modifications and data .^anomalies.. • * 

Cooperation between the contractor and the archivist results in not -only 

a useful repository for data, but also 'an improved research effort: . ' 

■ Before addressing the requirements/ that archiving places oh the contractor. 

we will reiterate and emphasize .the majqr points of th^discussion above 

to- orient the contractor appropriately to the archiving process. 

• The archivist depends on the contractor for all information 
affecting the data. 

_t ' ; ; _ •' 

Cooperation from, the contractor in .delivering .products , to the 
archivist is essential to the creation of a' thoroughly documented • * 
data archive. . • > 

• ' Whenever in doubt, the contractor should provide %he archivist 
with more rather than less* study Information. 

* * '* * 

i The information the contractor delivery to the archivist 
is consistent with the requirements of good research conduct 
and reporting. f 

• Throughput the ptudy, contractors should carefully record all 

deviations from the proposed methodology (i.e., sampling plans, *" 
obtained populations, instrumentation , - coding Notices , 'etc . ) . 

• • In addition to the formation of a data archive, the Joint efforts.' 
of contractor and archivist will help to improve the research 
process by encouraging and making available "more complete artf 
accurate accounts of the study. • - ' 

1 • 4 . 
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f -I* AH OVERVIEW 'OF ' CONTRACTOR BEQu3Ra!ENTS 



• The. information the archivist requires from, the contractor is" determined 
-by a set of standards which have been developed for the preparation of a 
_ data archive. These standards are described in detail in Volumes III and 

IV of the Archiving Methodology . Briefly, therV are three levels of archive - 
documentation: project, file, and" programmer . The contractor's role in data 
archiving is focused oh the project and file levels. ~~ 

A. PROJECT-LEVEL DOCUMENTATION 

Project-level documentation is the most global level; it is usually 
an archive user's 'introduction to a specific data archive. The goal of project- 
level documentation is to acquaint the fearer, in 4 single volume, with the 
study that generated the data in the archive and to provide the reader with ; 

such information as the study's purpose, its design,, its sampling strategy, * 

*■ p • 
its key results. ' It also describes all the substudies undertaken as 

. . J* ' * 

. part pf a project. Reading the project-level documentation will- help the ; 

potential researcher to decide whether the study may fit his/her heeds. 

. To. produce an adequate project-level document, the archivist' requires • ' 

- i \ 

the proposal and all overview documents generated by the. study. Since documentation 
at -this level is relatively general, very specific details of study modications ' 
are not required. . In fact, a pro jectrievel document would fail to fulfill 
its primary purpdse—hamely, to give the potential researcher a general under-, 
standing of the study— if all specific data- were included in it. This information 
is more appropriately presented in the descriptions of the individual fil e (s) 
generated by the study. 



A 
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B. FILE-LEVEL •DOCUMENTATION * 

• If the project-level document .suggests that. the study fits a researcher's ' 
needs, s/he turns to the file-levelj-doeumentatien for more specific detail. '_ 
A file-level document is written- for each separate data file or each group -of 
identically structured files. ' These documents are the analyst's guides to 

using the data files. Usually, data generated by a single 'instrument comprise 

- ~» _ • j, 

a single file. However," in the case of an instrument administered three 

times, the data ff&m each administration' may reside on a separate physical 

r--_ '•' 

file. If each administration produced, the^ame data, a single file-level 

document Is sufficient. Separate file-level ^documents are required when " 

' « V ' - 

instruments differ substantially from one administration to another. "" - 

- > c _ • • - . 

To produce file-level documents, the archivist requires precise information 
._•*_. _ *_ _ _ _ . ■ . \ 

about each file." Often, a stuqy consists 'of multiple. data collection efforts. 

■ -•■ j * • . 

For example, tlie Elementary i^Ucatibh. Voucher Demonstration (EEVD) surveyed 3 

-_ -_ ' '■ ■ ' \ 

parents, teachers, principals, etc. Each of these data collection efforts > 

relates to the overall purpose of the EEVD- in a slightly different way and 

- * !** ' _ . . ... ! * 

contributes- to the. answers of the .overall research questions f The archivist 
must understand these purposes .and their relationship to the overall project 
purpose in order to explain the data file adequately.. Similar detail is ' 
required about ^all "the remaining components of the file-^evel standard (unit 

----- -- •..•__#.' _: J_. _ 

of observation, scale, ..time information, sampling and target population v 
data collection method, data anomalies, etc.). * ' * 

The following jiectioh will enlarge upon these - .general contractor: re- " ' 
qdirements and provide a formal mechanism to,-aid the contractor "in delivering 
them to the archivist* I 
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• ThC Pr ° CeSS of ? ata - collection can be viewed chronologically; that is, ' 

• in terms of the considerations a contractor must make about a study while " 
„ ft 13 being pla f e5 ' Vitiated, and implemented and in retrospect when * 

it is completed. • When a study's data 'are intended for a data archive, the ' ; 
contractor must be jpire that the archivist is also interested in what is : ' 
occurring during each of these data collection phases, 'again, we emphakV 
that these considerations normally occur' in the course of -any research sfudy, 
-regardless of whether the data is to be archived. The* only "burden" archiving ' 
Places on the contractor is thafeiof communicating these decisions to the 
archivist. ' — • • 



A. eROSS-flEFERENCED INFORMATION FORM (CRIF) 

Although archiving requires little or no additional work for the "con- 
tractor; other than the communication of study information to the archiv^t, 
it is necessary for the contractor "to use' a mechanism to facilitate such r 
communication. We propose a "Gross-Referenced Information Form," (CRIF) be 
used as this mechanism because it- meets several criteria * 

* I , t 1 h ff b ? en . Signed to provide the archivist with 

w -5 info , rQation needed to reconstruct the study 

* on both fcte project and file levels. 

* Despite its ^pr^niiwness, CRtf 11 a slipis forms • 
completing it will not demand a great expenditure of the : 
contractor's 'time and energy. - 

* ''llJftt clea ^ cfnciie, ea^y-to- U3 e "fopa, >hich;lists and 
asKs the contractor for information on a set of element* 
corresponding to -all phases .of the study . The contractor 
responds by tising SRIF to r*f«r the archivist to 

'certain pages of a study plication.., ■ 



; *' The assumptions underlying CRIF are that information required to : 
'archive data can be found in the literature the study generated and that 

the contractor Is not being' asked for anything thit does ^ot already exist. 
. The information required to cbmplete the CRIF may come from many sources: * * 

the original study proposal, quarterly or interim reports, OMB clearance - 

statements* final reports, appendices, acrd other study documents, it is ' 

• . — • * 

important to note that the development of the documentation is the archivist's 

role; the contractor is being asked to identify the location of the information 



needed for the\ documentation. , . ; 

» At the beginning Of the study, the archivist supplies the" contractor 
with copies of -different versions df^CRIF* Consequently, the information 
the archivist ^quires will be 'known to the contractor far in advance of 
its due date. As the ^tudy progresses, the contractor can gradually complete 
the CRIf . Ah advantage of this; approach is that it can 4ielp the contractor 
determine if all the necessary information exists in the study's reports 
and documents. Used in this way* CRIF is a useful checklist which not only 
supplies necessary information to 'the archivist but also reflects -the essential 
components of a research study. If the contractor encounters great difficulty 
in completing the CRIF, this can be interpreted as an indication to the con- 
tractor that certain areas of the study have hot been adequately discussed 
in study documents, or, more seriousty," that * certain essential research con- 

si iterations have not been' made. # * - 

- i ■ • 

Several versions df^CRIF appear in the next section. The 'reader will ? . 
readily observe that the elements comprising CRIF are essential 'components 
of all 'research studies'; CRIF does not. disrupt the nature of the process, 
bufc*rather reflects it faithfully. 3 * * V ; 4 

/ - \ 



Oh the- next few pages, we present several versions of CRIF. Our reasons 
for providing these versions relate to'the kinds of information the archivist 
needs from the contractor to generate project- and file-leve} documents. 
The contractor completes both project-level and file-lever CRIFs, indicating 
where relevant information may be found. The file-revel CRIF differs from, 
the project-level CRIF because more specific detail is needed for file-level • 
' documents. When the data archive is established, researchers will use the 
file-level documents in their work with separate files. To work effectively 
with a file, these researchers wiirneed precise information about its contents^ 
source , .and format . - • ; 

.- . ■ '. - } ' . - 

In addition, certain projects are overarching research endeavors, 

.ccnpcocd of nultiple substudies.' Such lar^e-scaie otudiec require a project- 

* * 

level CRIF appropriately organized^ to present the location -of information 

, ' . _ 

^aboot the overall project and its component sutstudies. For this reason % two 

i * i > . . * ' ' ' 

versions or the project-level CRIF_ are given below. " - 



1 . CRIF: Single- Study 

The single-study project CRIF shown on the next .page (Figure 1) is a 
relatively straightforward listing of study components. When the contractor 
has cited the locations of information about each component, the archivist 
has the information required to generate a project-level, document'; The 
archivist will decide how much • information cited is necessary to explain the- 
project adequately. i . ^ 

2. CRIF: Projects Consisting of Multiple Substudles * 

The version of CRIF shown in Figure 2 Is used when the data being 
archived represent large scale research project involving* more than. one related 



ERIC 



FIGURE 1 



Cp&m fiefe 
* — - 

Si^dy Title: 
- / 



information Form: Single- Study Projects 



Rationale for Data Collection Effort 

1. - Purpose (s) of study 

_ _ _ 1 t 

2. Major research questions 

3* Historical/theoretical background 
Study hypotheses 



Major study variables 
Method of Data Collection 
Data Analysis 
Major Findings 



Location 
Document gage j 



Cross -Referenced Informal 



Study Title: 



FIGURE 2 

- * 

^ Projects Consisting of Multipl e Substudies 

7— I ■ — 

Location 

„ L_L • ... A 

- Document gage g 



' A; Ration^Le p for Project 

1. Purpose (s) of project 

2. Major research questions ; 
3; Historical/theoretical background 

'4. Major Hypotheses > 

Substudy I....K 



Substudy Title:- 

■ \ — 

A/ Rationale for Substudy 

1; Relationship to overall project 
• 2. Purpose (s) of substudy 

B. Substudy -Design 

C. Method of b&ta Collection 
b. Data 

E. Major Findings 



Is 



subsidy. By completing tijis form, the contracted tills the archivist where .' 
in -the Reject literature to find 'the information, necessary to produce a 
project-level document. • Such a document acquaints the reader with general 
.information about the project and the separate .data 'collection efforts (sub- 
studies) it encompasses, items lettered I- K and A E are completed' for each 
substudy*" within the project. , . : 
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3. CRIF* Individual Study Files 
i 



The future ^archive user requires more precise information on the "file-level 
because it is this documentation which wijl enable the user to actually work ; 
with|the data. The^RIF for individual study files (Figure 3) reflects this > 
heed for, greater specificity. - » ). 

In-this version Of CRIF, researeh components are organized under the three 
headings: initial, ongoing, and post-data collection considerations. These • 
chronological distinctions are made to elicit facts abotft any deviations from 
the original .data collection plans. As' with the -previous Versions of the CRIF, 
the contractor provides only -the location of* this information; it is up. to 
the archivist to interpret the information cited and decide what is needed to 
generate a complete file-level document. 

B. DATA COLLECTION PROCESS CONSIDERATIONS ; 

This Methodology maintains the viewpoint that every decision made in data . 
•collection in some way affects the data themselves. The purpose of this section 
is not to instruct the contractor in the conduct of research, but rather 
to call the contractor's attention to Certain issues Critical to archiving 
which are often neglected Or treated haphazardly in study reports. 



S 
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FIGURE 3 



Cross-Rfeferehbed Ihfor 



File IA...K 



Title: 



File Title: 



individual 



ties 



Initial Consideration^ 
A^ Rationale for Data Collection 
1; Purpose(s) . 



Location 



2. Relationship to other study files 
an<| to study as a whole 

*■ 

Unit of Analysis 



Page 0 



Design of Instrument 
1 . Validity 
"ft Reliability ! 

Sample . Characteristics 

* 

1. Size/characteristics of universe 

— - — *■ \ 

2. 'Target popoiation 



r 



Sampling Strategy 



? 



Time Information 



1 



Ongoing Study Considerations 

A: Time Information Modifications 



Location 



Document 



B; : Sampling S 



Modifications 



C . Interviews/Unusual Occurrences 

D. instrument Modifications" 

pQSt-Coiiectibn Considerations 
A. Scale 



B. Obtained Response 5 

. 

C. Coding Decisions 



D. : Data Editing/Cleaning 



E. Weighting of Data 



T^e f ollowing is .a brieJT description of the important elements in CRIF 
and ah explanation of how these* elements will assist future users of ,the 
archive, the elements are grouped chronologically into initial study con- . v 
siderations, ongoing collection considerations, and post-collection considerations. 



1. Initial Study Considerations 



information about initial study considerations are likely to be found 
in the enabling legislation, the request for proposals (BFPs), the proposal, , 

the contract, and early status reports on the qtudy, as well as in more "final 1 * 

* ' . * 

documents. They include ^ 

* 

• rationale for data collection: 

■ 

• study design; - • % . 

t 

• instrument design; 4 * 
- • validity; 

• ' reliability; * " * <= 

. • sample characteristics, size, target population; 

• sampling strategy; 

• time information.! ' i 
a. Rationale for Bata Collection , 

Every data collection effort is designed to meet some expressed 

* -i 0 

purpose (s). The reasoh(s) for obtaining the data should be stated "clearly, 
specifying the historical and theoretical' background of the issue under study. 
The specific theoretical stance which guided the study and the hypotheses "' 
it Bested are major considerations 'in constructing a comprehensive account * 
of ^why the data were collected. , • 



i 
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, • b.j Study Benign i * ; ' ' - ; ■ ' * • 

The study design grows out of the specific theory being tested; it 
* is a blueprint diagramming the way study variables real te. These relationships 
form the theoretical model being tested. The study design -identifies the 
variables- of interest or the independent/dependent .variables when appropriate. 
. The operational definitions of study variables and the unit bf analysis can 
„ also.be included here. , ^ 

Future archive users must be informed of the design of the study from 
which the data originated, sinc^such information e^iaihr^nTlSfen^g of 
the data in the context of the previous study.. This information will assist. 



the secondary analyst in determining areas of theoretical weakness. 
'. c. Instrument Design ^ . 

— - - - - 0 

■ Usually, survey data are collected with some type of data collection 
instrument. Non-survey data are obtained using a form or the output of 
• an administrative system, it is important to archive users to know how data 
were generated and, in the case of multiple administrations, to know what, 
changes, if any, were made to these instruments over time. 

d; Validity "• , 

L, V 
ent is one which, in fact, measures what it purports 

to measure. Valid instruments are essential tools of sound research. There 

are several types of validity: "face, content, criterion-referenced, and 

construct. Surveys are. most likely to strive only for face and ' ' 

.content validity. The former type of validity 'refers to the appearance of 

tSe instrument^&hile the latter concerns the adequate, sampling of a'olearly 

specified subject area. Criteridh-ref erehced validity is, established when 
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ah instrument is: shown to correlate well with other criteria of a trait,, 
attitude, skill, etc. Finally, construct validity 1$ the testing of the 
underlying thebry of ah instruments r o the extent that the theory is up- 
held* an" instrument is content valid.; * . •/ 1 

Information about the validation procedures used ahl their outcomes eh- 

.o — ■ - ■ « 

• . . _ _ . " • * r 

able the future researcher determine if further testing is needed to establish 

. » * - - _ - - - ■ , _ _• _>_ _/ - _ _ 

an instrument's validity. In cases where validation was not performed, future 

users may perform their own. j * 

e. Reliability . 

• • _ 

The Reliability of ah instrument refers to its accuracy, precision* 

ahd re^icabiiity as a measurement tool. As with validation procedures, 

if reliability indices were constructed , future users will be interested 

; in their results and the techniques used. Where several waves of data are 

being archived, the precisipn of . the measuring instrument must be taken into 

\ _ _ " . ' '_ _ ■ _ # . 

account vrhen comparing the results of bh§ -^ministration to those of another. * 

■•- * - L * ' 

■ f. Sample Characteristics } 

Size charac terist i cs of the univers e . ' The universe is the population 
f from which a sample is 'drawn. Secondary analysts heed this type of documentation 
to help them generalize their results to the correct population. 

- - v • 

. ^ * Target population . Tijis component describes the type of sample drawn, 
*its important characteristics, and the reasons for' choosing it. The target 
group represent the "goal" population toward which data collection efforts 
strive. A thorough description of the target population acquaints the archive 
User with this group. * > <■ 



g; Sampling Strategy, 

• • A sampling strategy is the specific selection criteria and methodology 
used to draw .a sample appropriate to a study's objectives. The description • 
of this strategy gives future researchers ah opportunity to evaluate the ' • 

.. . . . ■ ■ i 

s anple«s_8Ultablllty for answering_the r esea rc h questions the study poses .- 

• ' ■ < " r ' ' 
h. Time Information * '« 

As pas| of the proposal and initial studyj^cohsio^atiohs, the contractor 
prepares a data collection schedule. It specifies when the data will be 
collected. (month, season, year) and "now often (once, many times, oi* continually) 
The actual time of data' collection may explain some seasonal effects; the 

« - ' — - - — • - - , , 

frequency of data collection can suggest possible bias as a result of attrition 
and/or the possibility of correlated results owing to repeated measures. 

Data usually refer to a specific time period in either the present ". «^ 

or past.. This specific time period, the "data time frame," is not ' 

- ; - . • < i '_ _^s_ 

v always obvious, especially if attitude is being 5 measured. The data time 

frame should be clearly stated to users of the data. Often, a respondent 

is asked to express a presently-held attitude about some past event, 

phenomena, object or person, for example, "How satisified are you with the 

_ - ■ _* _ _ _ - _* 

special services which your child received iast^ year at school?" In such 

cases, the time frame is considered present. 

2 V Ongoing Collection Considerations 

. S*- _ _ ___ • _ -'_ : f 

Ongoing collection considerations might be called "modifications and 

- - - — _ . - * V 

problems," since most of these considerations become important when, for 
-practical reasons,- it is not feasible or possible to adhere to all of the 
initial study considerations i Included ip ongoing Considerations are 



• sampling strategy modifications? % 
•" m interviews and other unusual occurrences in data collection; 

_\ * - ._ :_. 

• instrument modifipations. 

^ as Time' Information Modifications ; : . : . 

•» „ * ■ «■ * 

We. discussed this factor as an initial consideration; we discuss it . -• 

* _ • 

again here because the data collection factors of "when?" arid "how often?" 
are sometimes a- concern throughout the collection process. Time schedules • 
are most likely to deviate because segments of the target 'population' are ^ 
difficult- to reach, interviews can take longer to administer than planned, 
mailed questionnaires may be returned later than expected, and many. other 
interruptions can occur. Significant deviations from the proposed data 
collection Schedule can profoundly, affect the^fesultant data, especially 
deviations in^ving a period of months or failure to conduct a proposed 
wave of data collection. * 

b. Sampling Strategy Modifications - " 

' $ _.. ._ • ■_. ■ ._ . " 

Studies can initially propose to draw a sample of a' certain size and 

type and. later modify the^larlier strategy. There may be many reasons for 

modification, among them, practical factors, such as inability to contact « 

* v= . •■ 

desired segments of the population, or recognition- $hat the original sampling 
plan was overly idealistic or excessively expensive. An alternative sampling 
strategy may be adopted because unanticipated substantive issues arise which 

require selecting a hew or larger set of respondents. The rationale for 

. - ■ _, * ; » 

employing a hew or modified sampling strategy and a detailed description 

of the alternative strategy should be included, since without this informalion, 



the archivist is unable to Adequately convey the rationale underlying the ' * 
data in. the archived file. . * , 

~'c. -Interviews and Other Unusual Occurrences in Data Collection , 
Interviewing is a common technique for collecting data. . Unlike the 
mailed questionnaire which is a relatively controlled stimulus for response,, 
the interview situation may encourage variation, primarily because of inter- / 
viewer interference or bias. Interviewer bias can contribute significant^ 
to response error and is difficult to control even with the most *deal training 
conditions. '.The interpersonal reaction between interviewer and respondent 
is another factor which can influence- the nature of responses. A host of 
other factors can affect the responses 'obtained, among them, length of inter- 
view; convenience of interview appointment for respondent; interviewer training 
and experience; language competency of respondent; cultural characteristics \ 
of interviewer and respondent; relevancy of interview to respondent; and . 
clarity of the questions asked. * 

When, conducting a study which uses the interview for data collection, 
the contractor must be aware of all of the above' factors and carefully note 1 • 
any factors which seem, to influence the data collection process, ft useful 
technique some studies employ is to ask interviewers to report to "a" super- ' 
visor periodically to discussjjroblems and any peculiarities encountered 
in the set of interviews completed. Since such problems and peculiarities 
are Important to the archivist, contractors can instruct the supervisor to 

* t- "... 

maintain records of them! Here, as' with Other deliverables, ,the contractor % * 
can rely on the advice that it is better to provide the archivist with* "more 
rather thankless" study information. Although much of this 'information may*' 
not ultimately be included in the archive, it will enable the archivist to 
make more informed decisions about what is appropriate explanatory material. 



*ny other unuiual mm or p&U* mm ooour during th. data collection 
T" ° f ***** M * « «*r«* »*. interSil s i entS( suaK . . 

" * ^ nai6 « «* * met* oh data collection; ' 

"Externa!* oceurahoee, aueb ae atrifcea, i'ttiUl^,- «Wty aeti^or, , 
ganiaational change*, natural or «en«ic diaaatera, a* other « 
to aniironient iay be relevant to the iaua* baihg instigated' by the/atudy. 
. d. instrument Modification < : * '\ 

*Kh a aurvey ls adsinlatered in Vavy,' lt,M,,beccie evident thatW \ 
in the data collection imtmm^m^ aAronbXaao.e tb.r'aapohdenta. 
In addition, iasuea are.anbjeot.to change. Specially in longitudinal atudies 
and in lsrge,acal,..en*iro 1 »enta 1 studies; For theae reaaona, it^ta,.* - 
delatad.-revia^. or added. Since the instruct, are the tools «le« general 
the data, it is. essential that all instruaeh^difications be 'cleariy described. • 
an explanation of. the HaSU, f* chahgea t„ V ti» atudy ihatnSiSta can- ; 
.help explain why the data Bay differ from adilniatratibo.fo adiiniatratioh 

3 - foat^bli eotion Coha ia^rrttnr,. ' . : ' • ' " 

- These are the considerations made after the data have been collects* 
They mate to how the data ware to be analysed to answer" the Search Rations . 
6f course, all of these considerations were also made in the beginning of 
the study. However, the reality of t*e Elected data may no* be consistent > 
with origin expectations for it." Hence, many of tha following issue, »* ' * 
reconsideration; others arise for the first time: ' - 

• scale; *. ^ 

• obtained response; ' . " 
* * coding decisions: . . . • 



t - 

• data editing 'and cleaning; * - * 

* ■ ; ' (* ■ 

• weighting of data; ' 



• analysis processes; . - V 

i proposed techniques; i 



f inde* constructions 

# actual statistical' techniques; 

• data anomalies and limitations; 



major findings; 



s 



• f ile processing considerations, 
a. Scale 



Scale is the size of the sfeta . There are two components of size: 1) the 
^ number of rcases, that is, the actual number of records on the file; and 2) 
the number of variables collected, that is, the number of data fields'each 
record contains. Scale informatidf is important to the archivist, since < 
^secondary, analysts and^ther 'users of the archive w*ll have to determine 
whether a file is an appropriate size for their needs. ♦ * 

b. Obtained Response ^ S - 

^ J he deset *i*i«* of the characteristics and size of tie data* file can 
differ substantially from the_intended target population. The obtained 
population represents .the true contents .of the data file. The differences 
between the obtained and target populations most •.commonly occur because 
of problems encountered to data, collection. Some typical problems are that 
. respondents were hot at home, they refused tb participate, o r questionnaires 
^ were mailed to, their old addresses. Respondents ^.misinterpret questions 
wnieh necessitate the removal of their resppnses from the sample. 
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iii- significant dianrepancies betMen the 0 ^ aned wa ta . get „. bl . .. ms 

Should be reported to the archivist: «i* iifoSatioo «ii help fcure re, • : 
searohers understand any bi ae to data due to nenresponee. . 

c Coding Decisions : s * 

' CbdiBg Practices and decision* abWfiow to treat open-ended 
in particular, vitally affects the mm* of the data > Contractors should • 

delete the appropriate sections of "-he CKF at the time tb r , - 

ar-ade. If tWs „ dene , j s , archl , lst ^ „ ot - „ ^ __ ^ ^ 

dories ^individuals involve* in the stud y and win « Save to engage 

in an ex sost facto » »»dy decisions; instead, the archivist 

will receive complete "on the spot* reports It SSSm. K 

p leporcs. it cannot be overly emphasized 

that contractor should *» on ; the CRIP as the study progresse6 . j^-- ^ 
archiving.,*, shown tbat researchers, busy with present projects, bave great 

'difficulty remembering the details of studies in f J 

i _ >■ ■ «uaies in which they were previously 

involved./.. 

* 

Data "Editing and Cleaning- 

Dlta editing and cleaning consistency checks performed on data include 
7 SyntaCti ° ^ ^antic. checks and the Mutation of errors or missing " ' 
- values. Syntactic checks concern the for. and charactertistics of an individual 
data field and insure that each field conforms to individually defined ■ : 

characteristics. For example » i ssr — ■ . . 

exampte, IT a SO field were specified as alphabetical, 

with acceptable values of *M;". ep^S or "blank;'' a syntactic check would be - 

used to MeS rt*l» tfift io other vai«& j, tM ^ 

check investigates relationships between two or more variables and insures 

that data within a record !, consistent .hd reasonable. Thus, an example ■ 
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of . seS.Si.i6 check mM Se-iS^ tMt e student «rit h , SaMTof .i ^' 
have an AGE between H and 8. 

7 * . After errors of iissing values haVe been defected, the contractor, iust ' 

.decide ho S tney should be treated. Tbe .contractor can choose to leave the 
• s decision up tcthe user of the data or tc eaploy none sort of correction 
^ocedure,^^ 

- automatic data correction, if hew valine *n vis a±- * ' 

• new ? aiues or Mutations have been used to 

replace' old or fissing values ± n the data,' the contractor should cite «, 

„ CR1F l0Cati0a " thiS <° f °™"on « «. study literature, 

.Future archive Users reauire thi* inf^»i< 

. require this information in order to gauge the appro- ° 

Priateness and the magnitude of the imputations made i« the data. 

' Inf0rmati0n ° n thC Ch6Cks on the data and their results will 

inform the archive user :that the range of values is legitime oT .ignal 
the existence of problems within the data. 

e. Weighting of Data 

Buring the course of the stuJy, the researcher m K estlBate ?' 
population parameters or establish feStionships r^niriBg ^_ 

! S1 " 0e Sa " PUS ^ **** - * -e substrata cf the population 

that hare been over, of undef-sa«pl S d, fteights can be assigned to cases 
that is, individuals ^ ba Weighed acre or less heavily tb ap ^ te 
the characteristics of the population. ■ 

* particularly intent concern with wajStii, m secondary analysis ' 
U the population s «« this lighting system «, devlseo. The contractor ' 

m " St "^"^ lndii " e to « ^tended for na«onal, state 

or local estimates. ' ~ 

future «*r» of an archive ^taW data designed to obtain- national * 
estimates are .pared time **, energy if the, i^ i^ « m ' 



ERIC Q 



30he.es ployed in previous studies. However, is popmstlbn characteristics- 
change over tim, future users should cautiously apprbabh previ6„sly used 
weighting scheoes, particularly if the seheoes Were developed several years 



prior. 



f. Analysis Process Considerations- * 

As With t«e ; oonsiderations cade for data collection, the archivist is ' 
interested'not only in proposed data analysis technics, hut also deviations 
fro. the initial analysis plan*. Of particular i.pcrtanee to the. archivist 
are anoaalles found .in tne/dataas a result of the analytic process. - The 
actual-results of data analysis are net of primary interest t, the archivist 
unless these results point to' the existence of irregularities within the . 

cjata. * 7 ' - 

* • ... * 

The following is 'a' description of tnecglF eleoents which apply to data' 



S. • 

i 



. " P " r °°r" ■ Ba « m its =ost general usage, involves 

nuncrlcai description, estimation, and inference with regard to,., ict of data. 
Parous .statistical technics c dn ha perked .on a dat, set, They range in' 
complexity fro, Staple sugary statistics, xeaaure of central tendency and 
dispersion, to »ore cplex. Bultiple regression technics. Eegardless of ' 
the technique e^yed, the ultMate purpose of data analysis is to answer » 
the questions posed by the research study.. ' 

* P 

Knowledge of the types %f »tatistical^alyses planned for a data sit 
will enable. future researchers to decide, if additional Kind, of analyses 
could be used to enrich and . Extend the worx of the original contractor and 
to provide- further insights ihto theW Scrutiny o7 statistibal techniques 



can uncover possible misapplications of techniques arid serve as a check on 
the results of the original study. 

Extensive descriptions of statistical methods used are hot required, 

since such information is of relatively limited value to future users of 

__,<. _. • 

the archive. If users require detail about the statistical methods employed 

in the study, this information can be obtained from the final study report 1 . 

>J 

, index Construction. In addition to data analyses , ; the original researcher 
•• team ma y build indices which are measures resulting from the combination of 
the values of several items or variables. An index is usually a more 
efficient gauge of a 'construct than is a single item or variable. 

These data manipulations are important in a study, since they explain 
the' theoretical structure of the variables. As with data analysis, the 

indices employed should £e described briefly; detailed accounts of these ' 

. ■• >• * • 

data manipulations are not required . , • " ■ 

Actual St atistical Techniques Performed, During the course pf a study, : 

additional analyses may be undertaken. This decision may be made because 

the results obtained using the initially proposed techniques suggest that 

more analyses be performed, or because the nature of the original expectations 

about the data have changed*: To provide an accurate account of the statistics : 

used in the study, information, about the deviations from the previously 

C r % ■ _ : 

planned techniques ate used. , . * 

-_ ' - ' . ' , l 

Data Anom a lies and Li mitat i ons . Problems and anomalies arising from 

the data collection effort can include incomplete samples, anomalies in 

the instrument or its accompanying instructions , and adjustments made to *^ 

the data after collection. If problems, anomalies, and the strategies for , 
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treating them are not clearly explained * the result may be improper inter- 
pretation .of the data. Problems and anomalies can be uncovered during data 
collection * editings 'and cleaning or 'as a result-of the data analysis 
procesS. To properly document the data for future archive users i the 
archivist requires this information. 



g. Major Findings 

The findings of interest to the archivist are those indicating whether 

i 

or not the study supported the hypothesis, these findings are recounted in 
the project-level documentation, which gives the potential user an overview 
of the project. The significance of these findings and their ^relationship 
to theory are not required by the archivist. Future researchers who desire 
this information can consult the final report cited in the bibliography 
accompanying each study. ? 
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